City of San Francisco Trees

Imagine you've been commissioned by the City of San Francisco to tackle a problem they've been having with local flora. The parks department has taken extensive documentation of the city's trees since the 1970s - what species are growing, where they are, who they're maintained by - amassing a dataset of over 200K trees in that time.

The funding for that project has recently been called into question, and the City Board needs to see its value in reapproving funds for the following year. Stakeholders have raised several concerns over the past few years, and your job is to use the data to answer them. Good luck!

Jupyter Notebook

First things first, let's get some terminology straight.

Jupyter notebooks have a few special properties that make it ideal for work with data:

Anything you can do in Python, you can do here!

  1. Write a function that takes a string as input, and does something to it
  2. In a new cell, call the function and test it out

Importing packages

We use the pandas package to easily work with data as tables.
The numpy package allows us to work with some other special data types, like missing values

We'll rename these as pd and np, just so its easier to refer to later on

Importing data

For this semester, we'll typically work with data in tabular format, the type you'd be used to in an excel spreadsheet. Data files saved in this format will usually have a .csv file ending, short for comma seperated values.

For example, a CSV file could look something like...

tree_number, species_name, address
312, Magnolia grandiflora, 2828 Divisadero St
124, Melaleuca quinquenervia, 485 Union St
912, Pittosporum undulatum, 47 Vicksburg St

To import this, let's use the pd.read_csv() function:

Here, we've saved the data to a dataframe object named trees

DataFrames contain our data in little "spreadsheet"-like structures. Whatever manipulations you can think of doing to the data, you can likely search how to do

Exploring dataframes

Let's take a look at the data. We'll use the function .head() to read in the first 5 rows

How big is the dataset? .shape returns a tuple with the dimensions as (rows, columns)

Let's try to understand our data a bit better.

Show the biggest trees by sorting the dataframe:
Note: dbh records diameter of the tree base

Subsetting

Subsetting is a super helpful tool. We'll take a look at this more depth in next week, but for now, here are the basics:

We can filter rows from a dataframe based on some condition

How would you show only trees north of Golden Gate Park (latitude > 37.77285)

Hint: Same way as matching if statements in python, mirroring the syntax above

Data Manipulation

What is the average diameter of the Evergreen Pear tree?

Visualization

First things first, let's import the package to help us visualize the data, plotly.

If this package isn't yet included, we can install it using !pip install plotly. More on this week 5.

Note that we're using the sub package of the broader package, called plotly express. This simplifies a lot of the more difficult steps

Plotly express has a broad range of options to play with, let's take a look at the documentation.
Do a quick google search to pull up documentation for px.scatter OR run px.scatter? in a Jupyter cell

Clearly, there aren't any obvious trends going on from this view. Let's add in some more parameters

Geographic Plots

The transportation department wants to know track any trees sitting on a road median, in order to quickly remove debris after a bad storm.